Integrating Guidance into Relational Reinforcement Learning

Reinforcement learning, and Q-learning in particular, encounter two major problems when dealing with large state spaces. First, learning the Q-function in tabular form may be infeasible because of the excessive amount of memory needed to store the table, and because the Q-function only converges after each state has been visited multiple times. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. The first problem is often solved by learning a generalization of the encountered examples (e.g., using a neural net or decision tree). Relational reinforcement learning (RRL) is such an approach; it makes Q-learning feasible in structural domains by incorporating a relational learner into Q-learning. The problem of sparse rewards has not been addressed for RRL. This paper presents a solution based on the use of reasonable policies to provide guidance. Different types of policies and different strategies to supply guidance through these policies are discussed and evaluated experimentally in several relational domains to show the merits of the approach.     

Exploiting transition locality in automatic verification of finite-state concurrent systems

In this paper we show that statistical properties of the transition graph of a system to be verified can be exploited to improve memory or time performances of verification algorithms.We show experimentally that protocols exhibit transition locality. That is, with respect to levels of a breadth-first state space exploration, state transitions tend to be between states belonging to close levels of the transition graph. We support our claim by measuring transition locality for the set of protocols included in the Mur verifier distribution .We present a cache-based verification algorithm that exploits transition locality to decrease memory usage and a disk-based verification algorithm that exploits transition locality to decrease disk read accesses, thus reducing the time overhead due to disk usage. Both algorithms have been implemented within the Mur verifier.Our experimental results show that our cache-based algorithm can typically save more than 40% of memory with an average time penalty of about 50% when using (Mur) bit compression and 100% when using bit compression and hash compaction, whereas our disk-based verification algorithm is typically more than ten times faster than a previously proposed disk-based verification algorithm and, even when using 10% of the memory needed to complete verification, it is only between 40 and 530% (300% on average) slower than (RAM) Mur with enough memory to complete the verification task at hand. Using just 300 MB of memory our disk-based Mur was able to complete verification of a protocol with about 10⁹ reachable states. This would require more than 5 GB of memory using standard Mur .     

Ontological reasoning in the design space exploration of advanced cyber–physical systems

Cyber–physical systems are becoming increasingly complex. In these advanced systems, the different engineering domains involved in the design process become more and more intertwined. Therefore, a traditional (sequential) design process becomes inefficient in finding good design options. Instead, an integrated approach is needed where parameters in multiple different engineering domains can be chosen, evaluated, and optimized to achieve a good overall solution. However, in such an approach, the combined design space becomes vast. As such, methods are needed to mitigate this problem.In this paper, we show a method for systematically capturing and updating domain knowledge in the context of a co-design process involving different engineering domains, i.e. control and embedded. We rely on ontologies to reason about the relationships between parameters in the different domains. This allows us to derive a stepwise design space exploration workflow where this domain knowledge is used to quickly reduce the design space to a subset of likely good candidates. We illustrate our approach by applying it to the design space exploration process for an advanced electric motor control system and its deployment on embedded hardware.     

无人机群目标搜索的主动感知方法

为提升蚁群搜索算法在规模大的栅格环境中对未知目标的搜索效率,提出基于蚁群算法的主动感知搜索框架。该框架通过应用历史环境信息来选择无人机的运动方式,并由无人机运动方式和感知域信息得到新的环境信息,从而实现无人机群的智能自动化搜索功能。新方法计算出一种具有探索偏好的未搜索概率,可使无人机搜索时偏向未搜索程度高的栅格,以此来提高算法的搜索能力。同时,以未搜索概率和信息素作为运动方式决策的依据来建立一种新的运动方式选择机制。该机制不仅考虑了目标可能出现的区域,又可兼顾未知区域,从而可实现无目标先验信息条件下的搜索过程。仿真结果表明,此算法在规模大的栅格环境中,与现有算法相比具有更高的搜索效率,并且得到的目标分布信息将更加全面。     

矿产勘查生产要素配置中政府和市场作用分析

矿产资源是经济社会发展的重要物质基础,是国家安全的战略保障。党的二十大要求“构建全国统一大市场,深化要素市场化改革,建设高标准市场体系”,强调“充分发挥市场在资源配置中的决定性作用,更好发挥政府作用”。本文分析了近年来矿产勘查生产要素配置现状,梳理了矿产勘查中政府和市场关系演变和作用机制,从充分发挥市场决定性作用、构建“有效市场”和更好发挥政府作用、塑造“有为政府”两个方面提出对策建议,以期为充分发挥政府和市场两个主体在矿产勘查生产要素配置中的作用提供一些思路。     

“实践体验式教育”在大学生党员中的实施路径探析

文章简述了"实践体验式教育"的内涵特点,讨论了在大学生党员中开展"实践体验式教育"的必要性,并对实施路径进行了探索分析。     

Automatic Mapping of Dynamic Office Environments

We present a robot, InductoBeast, that greets a new office building by learning the floorplan automatically, with minimal human intervention and a priori knowledge. Our robot architecture is unique because it combines aspects of both abductive and inductive mapping methods to solve this problem. We present experimental results spanning three ofiice environments, mapped and navigated during normal business hours. We hope these results help to establish a performance benchmark against which robust and adaptive mapping robots of the future may be measured.     

塔里木盆地构造特征与油气聚集规律

塔里木盆地是一个由古生界克拉通盆地和中、新生界前陆盆地组成的大型叠合复合盆地,具有古老陆壳基底和多次沉降隆升的复杂构造演化历史。古生界油气聚集受克拉通古隆起和斜坡构造控制,中、新生界油气聚集受喜马拉雅期逆冲构造控制。此外油气分布还受油气系统、区域盖层、断裂及不整合等因素控制。盆地具有多套烃源岩、多个油气系统、多套储盖组合、油气多源多期多种类型的特点。储集层条件好、天然气资源丰富和整体勘探程度低是在盆地进行油气勘探时不可忽视的三个特点。塔里木盆地已成为我国三大天然气区之一,油气勘探前景广阔     

三维线圈测量椭圆率

为提高电磁感应勘探的分辨率,需要对总的椭圆率进行精确测量。文章总结了传统的测量方法,指出它们有难以克服的缺点。通过分析合成磁场的空间特性,提出了一种三维线圈的结构,并给出了以结结构为基础的数学计算过程。从而克服了传统方法的缺点,为高分辨率电磁勘探开辟了一条新的思路,并且椭圆率计算方法也有其理论价值。     

川东开江区块渡口河温泉井构造复杂钻井事故剖析

在四川勘探钻井重点探区温泉井,渡口河两构造钻井中,速度慢,井下事故复杂频繁发生,严重制约该地区的勘探速度,通过分析江１,渡２,渡３,渡４和温泉４井的经验教训,建议加强对该探区浅层气的研究和认识;从蝇设计入手,增加设计前的地质情况调查,尤其是钻井用周边环境状况的调查,提高对井下,人身,地面设备的安全意识,控制上部井段清水浸泡时间,加强堵漏治工艺技术的攻关,提高固质量,加强固井工程管理。